The present invention relates in general to video image processing and in particular to sub-band decomposition of image signals.
Many methods used for local noise reduction are based on the adaptive application of linear local operators to the individual pixels or the signal values of the pixels. Such methods are described by Le, J. S., in the article entitled “Digital Image Smoothing and the Sigma Filter”, Computer Vision Graphics Image Processing, Vol. 24, pp. 255–269, 1983; in the publication by Tekalp, M. A., entitled “Digital Video Processing”, Prentice-Hall, 1995; by de Haan, G., T. G. Kwaaitsaal-Spassova, M. Larragy, and O. A. Ojo, in the article entitled “Memory Integrated Noise Reduction IC for Television”, IEEE Trans. on Consumer Electronics, Vol. 42, No. 2, pp. 175–181, May 1996; and by Schröder, H. and H. Blume in the publication entitled “Mehrdimensionale Signalverarbeitung [Multidimensional Signal Processing]”, Vol. 2, B. G. Teubner, 2000 for example. This principle of applying linear local operators to individual pixels can be combined only to a limited degree with the image model present (problems occur in particular at the edges of the image) so that the efficiency of this method is limited.
The method described by Lebowski, F. in the publication entitled “Bildschärfeverbesserung von hochaufgelösten Festbildern [Image Definition Improvement in Still Images]”, VDI Verlag, 1993 uses hierarchical stepwise processing in which threshold values depending on the frequency proportions are set in the signal to control noise reduction.
There are also methods based on sub-band decomposition of the image signal followed by processing of the sub-bands. In past years many papers have centered on the design of appropriate sub-band decomposition methods. The linear decomposition methods include simple linear separating filters, described by Rossi, J. P., in the publication “Digital Technique for Reducing Television Noise”, SMPTE Journal, Vol. 87, pp. 134–105, March, 1978; there are also pyramid decomposition methods described by Burt, P. J. and E. H. Adelson, in the publication “The Laplacian Pyramid as a Compact Image Code”, IEEE Trans. on Communications, Vol. 31, pp. 532–540, 1983, and methods based on wavelet decomposition of the signal.
However, linear decomposition methods have only limited usefulness for noise reduction in image signals because of their properties in the edge area. The goal of sub-band decomposition is extraction or filtering out of image elements and features with special properties. When band splitting occurs, the use of linear filters leads to sub-bands that differ in the frequency components they contain. This type of signal splitting is cumbersome, particularly in the case of sub-band decomposition for noise reduction, as typical image contents consist of a great many edges as well as other elements, and these edges in turn are composed of many different frequency components. The result is that the signal components of an edge enter many different bands, making it difficult to distinguish between noise and signal components. Because of this, either no noise reduction is accomplished in the edge area or signal components of the edge are recognized as noise and are accordingly removed from the signal.
These problems do not occur in nonlinear sub-band decomposition in which the edges of an image signal are not broken down into many small portions but stay together in one sub-band. Such nonlinear sub-band decomposition using a filter bank with median filters is described for example by Salembier, P. and M. Kunt, in the publication “Size-sensitive Multiresolution Decomposition of Images with Rank Order Based Filters Signal Processing”, Vol. 27, No. 2, pp. 205–241, 1992 and explained further with reference to FIG. 1. The basic design and function of a median filter is described for example by Helmut Schönfelder, in the publication “Digitale Filter in der Videotechnik [Digital Filters in Video Technology]”, Drei-R-Verlag, Berlin, 1988, pp. 125–127. Quadratic median filters for image processing are described for example by Rosenfeld, Kak, in “Digital Picture Processing”, Academic Press, Inc., 2nd edition, 1982, pp. 261–263.
Filter 10 illustrated in FIG. 1 has K quadratic median filters 12 of increasing sizes, which each receive the image signal S(x, y). The K median filters 12 have different filter lengths or filter sizes so that when determining the output signal value of a given pixel, the filters 12 consider the signal values of different numbers of pixels from the environment of the given pixel. The filter length and filter size increase with the number K of median filters 12 used.
At the output of the filter 10, K+1 signals are available, corresponding to decomposition of the image signal into K+1 sub-bands. These K+1 signals comprise the output signal of the K-th median filter 12, difference signals formed from the output signals of the adjacent median filters 12, and a difference signal formed from the original image signal and the output signal of the first median filter 12.
The signal at the output of each of the median filters 12 is characterized in that it contains the original image signal in which image features with half the size of the median filter 12 are suppressed. In addition, the noise output in the image signal is reduced by the use of the median filters 12. For the filter 10 illustrated in FIG. 1, this means that the signals at the output of the median filters 12 contain fewer and fewer image features as K increases and are increasingly free of noise. Formation of the difference of the output signals of the median filters 12 thus leads to output signals of the total filter 10 that contain only image features that can just pass the smaller of the two median filters 12 from whose output signals a difference signal is formed but which are already suppressed by the larger of the two median filters 12. The output signals of the total filter 10 also contain noise portions, which contribute in this size range, but which are strongly limited in their amplitude.
In summary, at the output of the filter 10 illustrated in FIG. 1 are signals that each represent a sub-band of the frequency spectrum of the image signal and that each correspond to image elements of different sizes and still have noise components with a low amplitude when the image signal is noisy.
The disadvantage of the band splitting illustrated in FIG. 1 using parallel median filters 12 of different lengths is that the size of the filter mask increases with the number K of the median filters 12, namely with the number of stages. It has been proposed for example that the size of a quadratic filter mask be allowed to increase exponentially with the number of stages. However this makes determination of the median values rather expensive.
To reduce this expenditure, a proposal has been made to reduce the necessary filter size and data volume by stepwise sub-scanning of the signals. Non-linear pyramid decompositions are described by Cha, H. and L. F. Chaparro, in the publication “Adaptive Morphological Representation of Signals: Polynomial and Wavelet Methods”, Multidimensional Systems and Signal Processing, Vol. 8, pp. 249–271, 1997, and by Donoho, D. I. and T. P. Y. Yu, in the publication entitled, “Nonlinear ‘Wavelet Transforms’ Based on Median Interpolation”, SIAM Journal on Math. Anal., Vol. 31, No. 5, which are based on median filters and morphological operators. However, noise reduction based on pyramid decomposition has the disadvantage that these methods are not shift-invariant so that the resulting image contains phase-dependent noise. Moreover, a polynomial approximation is proposed in the cited references for synthesis of the pyramid signals, which has the same problems in the edge area as is the case with the linear sub-band decomposition method.
Other methods for nonlinear sub-band decomposition that have better properties than linear sub-band decomposition filters in the edge area include not only sub-band decomposition using median filters but also wavelet decomposition with signal-adaptive lifting, described for example by Claypoole, R. L., G. Davis, W. Sweldens, and R. Baraniuk, “Nonlinear Wavelet Transforms for Image Coding via Lifting”, submitted to IEEE Transactions on Image Processing, 1999; by Heijmans, H. J. A. M. and J. Goutsias, in the article entitled “Nonlinear Multiresolution Signal Decomposition Schemes: Part II: Morphological Wavelets”, IEEE Trans. on Image Processing, Vol. 9, No. 11, 2000; and by Piella, G. and N. J. A. M. Heijmans, in the publication entitled “Adaptive Lifting Schemes with Perfect Reconstruction”, Research Report PNA-R0104, Centrum voor Wiskunde en Informatica (CWI), 2001. Because of the high expenditure and the properties of the sub-bands, these methods have limited usefulness for sub-band decomposition, namely dividing the image signal into various sub-signals that represent different frequency bands of the image signal.
What is needed is an improved technique for sub-band decomposition of image signals.